Changes

This PR updates the sqlglot requirement to allow the 27.0.0 release. This release includes support for placeholders in SQL, a relatively new syntax feature supported in DBR 15.2 (or Apache Spark 4.0).

SQL queries can be of the form:

SELECT 1 AS col1, 2 AS col2, 3 AS col3 FROM {sdf_system_columns} LIMIT 5

…where sdf_system_columns is passed as a keyword argument, for example:

sdf_system_columns = spark.read.table("system.information_schema.columns")
sdf_example = spark.sql("SELECT 1 AS col1, 2 AS col2, 3 AS col3 FROM {sdf_system_columns} LIMIT 5",
                        sdf_system_columns = sdf_system_columns)

Older versions of sqlgplot would fail on this with a parsing error, but now it's parsed correctly but returns a Placeholder AST node as the name of the table instead of a string. Given the complexity of handling this properly, for now I've chosen to preserve the old behaviour: the linter will mark the query as unsupported. (See es-1285042.py for an example of this.)

Linked issues

Closes #4203

Tests

• existing integration tests
• existing (+ 1 updated) unit tests